A method of correcting poor color tone of an eye image due to a photographic flash of a camera has been proposed. Note that poor color tone of an eye image is generally well known as a red-eye effect. The red-eye effect is as follows. That is, when a person or an animal such as a dog, cat, or the like is photographed using a photographic flash in r an insufficient illumination environment, the flash light that has entered the opening pupil part of the eye is reflected by the eyeground, and the capillaries gleam red. Since a person who has a pale pigment color (or a light pigment) has a higher pupil transmittance, i.e., a crystal lens, the red-eye effect tends to occur more frequently with such a person.
In recent years, digital cameras have been reduced in size, and the optical axis of a lens thereof tends to approach the light source position of a photographic flash. In general, as the light source position of the photographic flash is closer to the optical axis of the lens, the red-eye effect occurs more readily, and taking measures against this phenomenon is critical.
As one means for preventing the red-eye effect, pre-light emission is performed before photographing, and taking a picture after the pupil of the object is closed is known. However, this method consumes more battery power than normal photographing, and changes the expression of the object due to the pre-light emission.
Many methods of compensating for a red-eye image by correcting and processing digital image data photographed by a digital camera using a personal computer have been proposed.
The methods of correcting the red-eye effect on digital image data are roughly classified into manual correction, semiautomatic correction, and automatic correction.
With the manual correction method, the user designates and corrects a red-eye region displayed on a display using a pointing device such as a mouse, stylus, tablet, or the like, or a touch panel.
With the semiautomatic correction method, the user roughly designates a region where a red-eye image exists, and correction is applied by specifying a correction range of the red-eye image based on that information. For example, the user designates a region that surrounds the two eyes using a pointing device, or designates one point near the eye. Based on information of this designated region or designated point, the correction range is specified to apply correction.
With the automatic correction method, a correction region is automatically detected from digital image data and correction processing is executed without any specific operation by the user.
In the manual or semiautomatic correction method, the user must designate a correction region by some operation. For this reason, the user must display the vicinity of a region to be corrected of image data in an enlarged scale, and must then designate the correction region, thus requiring a troublesome operations. In case of a computer system which comprises a large-screen display device or the like, such operations are relatively easy to attain. However, on a device such as a digital camera, a printer, or the like, which comprises only a display device of several inches, operations for enlarging an image, and designating the correction region by scrolling the image are not easy at all.
In recent years, various proposals about automatic correction of the red-eye effect, which can obviate the need for troublesome user's operations and are effective for a device which does not comprise any large display device have been made.
For example, Japanese Patent Laid-Open No. 11-136498 has proposed a method of correcting pixels which form a red-eye image by detecting a skin color region from an image, and detecting pixels which are likely to form the red-eye image in the detected region. Also, Japanese Patent Laid-Open No. 11-149559 has proposed a method of detecting a skin color region, detecting a bottom peak region corresponding to the low luminance level of a pupil from that region, and determining the location of an eye based on the distance between the first and second bottom peaks. Furthermore, Japanese Patent Laid-Open No. 2000-125320 has proposed a method of specifying a red-eye region by detecting a skin color region, judging if the skin color region has features of a human face, detecting a pair of red-eye images from that region, and evaluating the distance and size. Moreover, Japanese Patent Laid-Open No. 11-284874 has proposed a method of automatically detecting whether or not an image includes images of red pupils, detecting, when the images of the red pupils are detected, their positions and sizes, and automatically converting red color pixels in the images of the pupils into a predetermined color.
However, the automatic correction methods of the red-eye effects which have been proposed suffer the following problems.
Detection of a red-eye region based on the result of skin color detection of a person or face detection using a neural network has a highly reliable detection result, but it requires a large memory size and arithmetic volume since a given broad range in an image must be referred to. Therefore, such detection is suited to processing using a personal computer which comprises a high-performance CPU that operates at the clock frequency of several GHz and a several-hundred MB memory, but it is difficult to implement such detection method in an embedded system inside a digital camera or printer.
In the embedded system in the digital camera or printer, the available work memory is as low as several hundred KB or several MB at most. On the other hand, the resolutions of digital cameras are increasing, and cameras having image sensing capability beyond 10 million pixels are available. In order to execute processing for detecting a red-eye region from such high-resolution image using a limited work memory, it is effective to decrease the resolution of an input image. For example, it is possible to reduce an input image of eight million pixels to a ¼ resolution, i.e., an image of two million pixels by subsampling that image at one-pixel intervals in the horizontal and vertical directions. In this case, the capacity of the work memory required to store an image is reduced to ¼. However, even when the image size is reduced to two million pixels, a work memory of about 6 MB (in case of 24-bit RGB data) is required to simultaneously hold the entire reduced image. Such storage capacity does not pose any problem in a personal computer or workstation comprising a large-capacity RAM. However, under the limited memory resources available in a digital camera, further means are required to reduce the required work memory size.
Not only the aforementioned automatic correction examples, but also most of the methods proposed so far specify a red-eye region by utilizing a feature that the red-eye region has a higher saturation level than a surrounding region. However, a determination based on saturation is not always suited to a person who has a dark pigment color (or a rich pigment). As is well known, when a pixel value is given by an RGB system, a saturation value S is given by:S={max(R,G,B)−min(R,G,B)}/max(R,G,B)  (1)where
max(R, G, B) is the maximum value of RGB components, and
min(R, G, B) is the minimum value of RGB components.
For example, as is apparent from the experimental results, the skin color regions of Japanese people are distributed around 0 to 30° as the hue angle (0 to 359°). Note that a hue angle of an HSI system near 0° is red, and it becomes closer to yellow with increasing hue angle. The magnitude relationship of RGB values around 0 to 30° satisfies:R>G>B  (2)
As described above, a person who has a dark pigment color (or a rich pigment) hardly ends up with a bright red-eye image compared to a person who has a pale pigment color (or a light pigment).
In consideration of these facts, the pixel values of the red-eye region and skin color region around the eyes of the Japanese can be estimated as:
Red-eye region: (R, G, B)=(109, 58, 65)
Skin color region: (R, G, B)=(226, 183, 128)
In the above case, the saturation value of pixels (to be referred to as “red-eye pixels” hereinafter) of the red-eye region is 40, and that of pixels of the skin color region is 43, i.e., these saturation values are roughly equal to each other. In other words, even when attention is focused on the saturation value, the red-eye pixels cannot often be specified, depending on the objects (persons) that are the subject to a photographing operation.